feat: Implement delta migration support for content mapping and entry…#1090
Conversation
… mapping steps - Added logic to handle delta iterations in the ContentMapper component, allowing for different behaviors based on the iteration number. - Updated the MigrationFlowHeader to adjust step IDs and button states based on the current iteration. - Enhanced the HorizontalStepper to maintain step completion status across iterations. - Introduced new constants for empty state messages in the content mapper. - Modified API calls to differentiate between new and old content types based on the iteration. - Map Entry step lists only content types that have entry mappings, with status icons derived from saved isUpdate state. - Made backend current_step iteration-aware (Map Entry inserts an extra step from iteration 2+). - Fixed entry-update: only remove entries that already exist in Contentstack (have a contentstackEntryUid); update them when marked isUpdate, otherwise create fresh — prevents data loss of never-migrated entries. - Updated the LoadUploadFile component to support MySQL connection details editing. - Adjusted the Migration page to conditionally render steps based on the iteration. - Improved the handling of empty states in the UI for better user experience during migration.
🔒 Security Scan Results
⏱️ SLA Breach Summary
🟠 High Severity - SLA Breached Issues (with fixes)Showing 4 issue(s) that have exceeded the 30-day SLA threshold:
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
❌ BUILD FAILED - Security checks failed Please review and fix the security vulnerabilities before merging. |
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
There was a problem hiding this comment.
Pull request overview
Adds iteration-aware “delta migration” behavior across the backend + UI, including a new Map Entry step (iteration 2+) and safer entry update handling to avoid data loss during re-migrations.
Changes:
- Introduces iteration-aware step numbering (5-step iteration 1 vs 6-step delta iterations) across UI stepper/side-nav and backend state machine.
- Implements filter-based content-type fetching (
filter=new|old) to split “new” vs “already-migrated” content types across iterations, and adds the new Map Entry UI step. - Fixes entry update logic to only remove entries from import data when they already exist in Contentstack, and adds unit tests for
updateEntryCli.
Reviewed changes
Copilot reviewed 21 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| ui/src/utilities/constants.ts | Adds Content Mapper empty-state copy for delta vs iteration 1. |
| ui/src/services/api/service.interface.ts | Adds iteration to migration response typing. |
| ui/src/services/api/migration.service.ts | Adds filter query param support for content type fetch. |
| ui/src/pages/Migration/index.tsx | Adds iteration-aware step list (Map Entry step) and CTA routing/disable fixes. |
| ui/src/context/app/app.interface.ts | Adds hasNoContentTypes flag for step-3 Continue gating. |
| ui/src/components/Stepper/HorizontalStepper/HorizontalStepper.tsx | Changes completion tracking to use persisted current_step. |
| ui/src/components/MigrationFlowHeader/index.tsx | Makes header CTAs/disable logic iteration-aware and adds step-3 empty-state gating. |
| ui/src/components/LegacyCms/Actions/LoadUploadFile.tsx | Adds tooltips to edit icons. |
| ui/src/components/ContentMapper/index.tsx | Fetches only “new” content types for step 3; improves delta empty state and step gating. |
| ui/src/components/ContentMapper/index.scss | Adds centered delta empty-state styling and formatting cleanup. |
| ui/src/components/ContentMapper/entryMapper.tsx | Refactors EntryMapper into standalone delta step (step 4) with CT list + entry table. |
| ui/src/cmsData/migrationSteps.json | Adds Map Entry step definitions and renumbers later steps. |
| package.json | Adds fast-uri dependency constraint. |
| package-lock.json | Updates fast-uri lockfile version/resolution. |
| api/tests/unit/services/updateEntryCli.service.test.ts | Adds unit tests for updateEntryCli (auth paths, errors, logging, exit codes). |
| api/src/utils/entry-update.utils.ts | Fixes entry-removal/update selection to prevent dropping never-migrated entries. |
| api/src/utils/entry-update-script.cjs | Removes a debug console.info. |
| api/src/services/runCli.service.ts | Uses iteration-aware final step number when marking migration complete. |
| api/src/services/projects.service.ts | Makes current-step state machine iteration-aware and adds MAP_ENTRY transition. |
| api/src/services/migration.service.ts | Makes Testing step update iteration-aware. |
| api/src/services/contentMapper.service.ts | Adds iteration diffing + `filter=new |
| api/src/constants/index.ts | Adds DELTA_STEPPER_STEPS + getStepperSteps(iteration) helper. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
🔒 Security Scan Results
⏱️ SLA Breach Summary
ℹ️ Vulnerabilities Without Available Fixes (Informational Only)The following vulnerabilities were detected but do not have fixes available (no upgrade or patch). These are excluded from failure thresholds:
Consider reviewing these vulnerabilities when fixes become available. |
…creen feat: Implement delta migration support for content mapping and entry…
🔗 Jira Ticket
CMG-997
📋 PR Type
📝 Description
What changed?
New "Map Entry" step (delta migration, iteration 2+)
Legacy CMS → Destination Stack → Map Content Fields → Map Entry → Test Migration → Execute. Iteration 1 stays the original 5-step flow (no Map Entry).Single-route, filter-based content type fetch
getContentTypestakes afilterquery param (new|old) instead of a new endpoint. On iteration 2+ it diffs the current vs previous iteration'scontentTypesMapper.jsonbyotherCmsUidto return the new-only or old-only slice. Iteration 1 is untouched.entryMapping, and derives each content type's status icon (green "Updated" / blue "Mapped") from the savedisUpdatestate so it persists correctly across reloads.Iteration-aware step numbering
current_stepstate machine is now iteration-aware (DELTA_STEPPER_STEPSadds Map Entry as step 4, shifting Testing → 5, Execute → 6 on delta iterations).project_current_step), and the side-nav (migrationSteps.json) all resolve step numbers by iteration.Stepper / navigation fixes
current_step, so navigating back to review a completed step no longer resets the progress bar, and completed steps stay navigable.Bug fix — entry update data loss
removeEntriesFromDatabasepreviously deleted every entry present in the entry mapper, even ones never migrated (nocontentstackEntryUid), silently dropping them. Now it only removes entries that already exist in Contentstack (have acontentstackEntryUid);isUpdateones are collected for update, the rest are left to be created fresh.Tests
updateEntryCliservice (basic-auth & OAuth paths, missing stack key, auth errors, stdout/stderr log-level classification, non-zero exit handling, warning mapping).Why?
Delta (re-)migrations need to map and update entries of already-migrated content types across iterations without losing data or re-mapping content that was already migrated. The previous flow had no Map Entry step, no iteration-aware step numbering, and a bug that dropped never-migrated entries during update.
🧩 Affected Areas
api— Node.js backendui— React frontendupload-api— Upload API serverdocker/docker-compose🧪 How to Test
Iteration 1 (fresh migration):
Iteration 2+ (delta / restart):
current_stepin the DB reaches 6 on Execute.Entry update data-loss fix:
7. Have an entry that was NOT migrated in iteration 1 — confirm it gets created this iteration (not silently dropped). Entries marked
isUpdatewith a Contentstack uid get updated.Tests:
8.
cd api && npm test—updateEntryCli.service.test.tspasses.Expected result: Iteration 1 = 5 steps unchanged; iteration 2+ = 6 steps with correct new/old content type splitting, persistent status icons, no progress-bar resets, and no entry data loss.
📸 Screenshots / Recordings
🔗 Related PRs / Dependencies
✅ Author Checklist
feature/,bugfix/, orhotfix/+ 5–30 lowercase chars.env/example.envupdated if new environment variables were added — N/A (no new env vars)npm test)updateEntryCli.service.test.tsREADME.md/ docs updated if behaviour changed — N/A👀 Reviewer Notes
getContentTypes?filter=new|oldparam + a per-iteration diff oncontentTypesMapper.json— no new endpoint.createStepper()(UI) andmigrationSteps.jsonall_steps(side-nav), plus the backendSTEPPER_STEPS/DELTA_STEPPER_STEPS.entry-update.utils.tsdelete rule — entries are removed from import data only when they have acontentstackEntryUid(exist in CS); never-migrated entries are intentionally kept to be created.